Aligning Superintelligence with Human Interests: A Technical Research Agenda
نویسندگان
چکیده
The property that has given humans a dominant advantage over other species is not strength or speed, but intelligence. If progress in artificial intelligence continues unabated, AI systems will eventually exceed humans in general reasoning ability. A system that is “superintelligent” in the sense of being “smarter than the best human brains in practically every field” could have an enormous impact upon humanity (Bostrom 2014). Just as human intelligence has allowed us to develop tools and strategies for controlling our environment, a superintelligent system would likely be capable of developing its own tools and strategies for exerting control (Muehlhauser and Salamon 2012). In light of this potential, it is essential to use caution when developing AI systems that can exceed human levels of general intelligence, or that can facilitate the creation of such systems.
منابع مشابه
Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda
The property that has given humans a dominant advantage over other species is not strength or speed, but intelligence. If progress in artificial intelligence continues unabated, AI systems will eventually exceed humans in general reasoning ability. A system that is “superintelligent” in the sense of being “smarter than the best human brains in practically every field” could have an enormous imp...
متن کاملAligning Superintelligence with Human Interests: An Annotated Bibliography
How could superintelligent systems be aligned with the interests of humanity? This annotated bibliography compiles some recent research relevant to that question, and categorizes it into six topics: (1) realistic world models; (2) idealized decision theory; (3) logical uncertainty; (4) Vingean reflection; (5) corrigibility; and (6) value learning. Within each subject area, references are organi...
متن کاملInferring Human Values for Safe AGI Design
Aligning goals of superintelligent machines with human values is one of the ways to pursue safety in AGI systems. To achieve this, it is first necessary to learn what human values are. However, human values are incredibly complex and cannot easily be formalized by hand. In this work, we propose a general framework to estimate the values of a human given its behavior.
متن کاملPower and Agenda-Setting in Tanzanian Health Policy: An Analysis of Stakeholder Perspectives
Background Global health policy is created largely through a collaborative process between development agencies and aid-recipient governments, yet it remains unclear whether governments retain ownership over the creation of policy in their own countries. An assessment of the power structure in this relationship and its influence over agenda-setting is thus the first step towards understanding w...
متن کاملGovernance for Mobile Service Platforms: a literature Review and Research Agenda
Mobile service platforms are IT-based marketplaces that have become the source of competitive advantages. Aligning the interests of stakeholders by establishing effective governance mechanisms is central to the success of mobile service platforms. This phenomenon ignites research in many disciplines, which results in a fragmented understanding of mobile service platforms. This paper is a first ...
متن کامل